For years the model performance in machine learning obeyed a power-law relationship with the model size. For the consideration of parameter efficiency, recent studies focus on increasing model depth rather than width to achieve better performance. In this paper, we study how model width affects the Transformer model through a parameter-efficient multi-path structure. To better fuse features extracted from different paths, we add three additional operations to each sublayer: a normalization at the end of each path, a cheap operation to produce more features, and a learnable weighted mechanism to fuse all features flexibly. Extensive experiments on 12 WMT machine translation tasks show that, with the same number of parameters, the shallower multi-path model can achieve similar or even better performance than the deeper model. It reveals that we should pay more attention to the multi-path structure, and there should be a balance between the model depth and width to train a better large-scale Transformer.

I've got some of these perceptions and systems technology and emergent behavior pieces that are relevant to this, and I'm smart enough to apply the necessary things. So, while I've got people that invested $20 million in my company, I'm not telling them that I'm likely to have the breakthrough for artificial general intelligence. Instead, I'm saying there's a non-negligible chance that I will personally figure out some of the important things that are necessary for this. The emergence of artificial general intelligence in a way that can impact the economy really is a'change-the-world-level' event, where this is something that reshapes almost everything that human beings can do. This is something that is almost the largest scale that you can think about.

Using LiDAR to get one step closer to fully automated vehicles and how Tesla is trying to accomplish the same goal but going different paths. LiDAR, radar, and sonar all sound so similar they got to have something in common right???? That was my first thought when I first heard about LiDAR. After a quick google search, I found out that LiDAR, radar, and sonar are actually very similar to one another. All 3 of these have the same goal: creating an accurate 3D map of their environments.

Artificial Intelligence (AI) for social good is a field of work which, broadly speaking, uses AI to make the world a better place. I had a chance to interview two leaders in the field, Dr. Bryan Wilder, who recently received his Ph.D. from Harvard (and will be joining the faculty at Carnegie Mellon next fall) and current Harvard Ph.D. student, Lily Xu. Both Bryan and Lily have been advised by Dr. Tambe, Gordon McKay Professor of Computer Science and Director of the Center for Research in Computation and Society (CRCS) at Harvard University and Director of AI for Social Good at Google Research India. While Bryan and Lily are both working at the intersection of AI and social good, they arrived at this junction via different paths. Bryan was studying computer science and looking for a field to apply his knowledge; his search led him to public health.

Time travel has been a beloved science-fiction idea at least since H.G. Wells wrote The Time Machine in 1895. The concept continues to fascinate and fictional approaches keep coming, prodding us to wonder whether time travel is physically possible and, for that matter, makes logical sense in the face of its inscrutable paradoxes. Remarkably, last year saw both a science-fiction film that illuminates these questions, and a real scientific result, spelled out in the journal, Classical and Quantum Gravity,1 that may point to answers. The film is writer-director Christopher Nolan's attention-getting Tenet. Like other time travel stories, Tenet uses a time machine.

Recently I came across an interesting Paper named, "Deep Ensembles: A Loss Landscape Perspective" by a Laxshminarayan et al.In this article, I will break down the paper, summarise it's findings and delve into some of the techniques and strategies they used that will be useful for delving into understanding models and their learning process. It will also go over some possible extensions to the paper. You can also find my annotations on the paper down below. The authors conjectured (correctly) that Deep Ensembles (an ensemble of Deep learning models) outperform Bayesian Neural Networks because "popular scalable variational Bayesian methods tend to focus on a single mode, whereas deep ensembles tend to explore diverse modes in function space." In simple words, when running a Bayesian Network at a single initialization it will reach one of the peaks and stop.

Years ago in college, I took a course on the Philosophy of A.I. and Machine Learning. This course had me thinking about computers in ways I'd never thought of them before--not as abstract, magical black boxes, but as mechanical devices using simple rules. Understanding how these simple rules lead to complex processes is the key to understanding machine learning. In this article I want to demonstrate the basics of Machine Learning -- one of the more advanced and cutting edge areas of Computer Science. And as in the previous articles of this series, we're not going to need any computers.

Future Apple smart speakers, like HomePod, might be much more data privacy focused. Apple has reportedly paid $200 million to acquire Seattle-based artificial intelligence company Xnor.ai, The purchase is one of many for Apple, which has become adept at vacuuming up tech startups, but it also gives us a glimpse into the company's thinking when it comes to future devices. Xnor.ai's work on hyper-efficient, low-power AI that doesn't require powerful processing or a connection to the cloud (processing locally on-device instead), neatly slots into a few areas Apple is currently working on. Whilst Apple hasn't - and doesn't typically - comment on why it acquires certain companies and how they fit into its future roadmap, we can speculate on how Xnor.ai's work fits in to the master plan.

Six of the eight largest public cloud providers worldwide–Alibaba, Baidu, Google, IBM, Microsoft and Tencent–have been investing heavily in quantum computing research and development (R&D). AWS, by far the largest cloud provider, had been completely absent from the quantum computing discussion until this week. The holy grail for cloud providers is to find a hardware solution that accelerates machine learning and artificial intelligence by orders of magnitude. There are two ways to improve machine learning at scale: build specialized architectures using today's design tools or find a completely different path. Quantum computing is everyone's big bet for the completely different path.

Incompleteness is a common problem for existing knowledge graphs (KGs), and the completion of KG which aims to predict links between entities is challenging. Most existing KG completion methods only consider the direct relation between nodes and ignore the relation paths which contain useful information for link prediction. Recently, a few methods take relation paths into consideration but pay less attention to the order of relations in paths which is important for reasoning. In addition, these path-based models always ignore nonlinear contributions of path features for link prediction. To solve these problems, we propose a novel KG completion method named OPTransE. Instead of embedding both entities of a relation into the same latent space as in previous methods, we project the head entity and the tail entity of each relation into different spaces to guarantee the order of relations in the path. Meanwhile, we adopt a pooling strategy to extract nonlinear and complex features of different paths to further improve the performance of link prediction. Experimental results on two benchmark datasets show that the proposed model OPTransE performs better than state-of-the-art methods.